Description

From #1697 (comment) @GraemeWatt notes:

The Accept headers will be ignored if archive_url is a HEPData URL (ending in ?view=true) that returns the content directly. The Accept headers will only become relevant when archive_url is a HEPData DOI for a resource file after we redirect to a landing page. We're still working on the relevant PR (HEPData/hepdata#412) after it expanded in scope. The default Accept header is */* for Python requests (and curl), different from the default Accept values of most web browsers where */* has a lower weighting (quality value) than text/html. With our current implementation, if */* has weight 1 in a request to the landing page, the content will be returned directly instead of the HTML landing page. This means you don't strictly need to specify the Accept header with Python requests. But I'd recommend you keep the Accept header to avoid accepting content types that your download code can't process. To accept both tar and zip files, you just need to use headers={"Accept": "application/x-tar, application/zip"}. We will return a 406 Not Acceptable status code if a request is made to the landing page where the Accept header does not match the media type of the content.

Checklist Before Requesting Reviewer

• Tests are passing
• "WIP" removed from the title of the pull request
• Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

• Summarize commit messages into a comprehensive review of the PR

* Add zip headers to the accepted content headers for contrib.utils.download
* Amends PR #1697

Co-authored-by: Graeme Watt <[email protected]>